Sparse Nonlinear Support Vector Machines via Stochastic Approximation

Nonlinear support vector machines (SVMs) are more broadly useful than linear SVMs, as they can find more accurate predictors when the data under consideration have curved intrinsic category boundaries. Nonlinear SVM formulations based on kernels are, however, often much more expensive and data-intensive to solve than linear SVMs, and the resulting classifiers can be expensive to apply. This paper describes approaches for solving nonlinear SVMs on very large datasets. The approaches make use of low-rank approximations to the kernel (constructed using randomized methods), minimization of a reduced primal formulation using stochastic approximation, and recovery of a sparse classifier that can be applied inexpensively to new data.